Message Broadcasting for Vehicles

ABSTRACT

Various aspects may include methods enabling a vehicle to broadcast intentions and/or motion plans to surrounding vehicles. Various aspects include methods for using intentions and/or motion plans received from one or more surrounding vehicles.

RELATED APPLICATIONS

This application claims the benefit of priority to U.S. ProvisionalApplication No. 62/782,573, entitled “Intention Broadcasting forAutonomous Driving” filed Dec. 20, 2018, the entire contents of whichare hereby incorporated by reference for all purposes.

BACKGROUND

Automobiles and trucks are becoming more intelligent as the industrymoves towards deploying autonomous and semi-autonomous vehicles.Autonomous and semi-autonomous vehicles can detect information abouttheir location and surroundings (for example, using radar, lidar, GPS,file odometers, accelerometers, cameras, and other sensors), and includecontrol systems that interpret sensory information to identify hazardsand determine navigation paths to follow. Autonomous and semi-autonomousvehicles include control systems to operate with limited or no controlfrom an occupant or other operator of the automobile.

SUMMARY

Various aspects include methods enabling a vehicle, such as anautonomous vehicle, a semi-autonomous vehicle, etc., to broadcast motionplans to surrounding vehicles, such as autonomous vehicles,semi-autonomous vehicles, and/or driver-operated vehicles. Variousaspects include methods for using motion plans received from one or moresurrounding vehicles. In various embodiments, motion plans may include avehicle's trajectory and one or more descriptors associated with thevehicle and/or the vehicle owner and/or operator. In variousembodiments, motion plans may be used at least in part to control avehicle.

Various aspects include methods of controlling a vehicle that mayinclude receiving an intention message including a motion plan for avehicle transmitting the motion plan (the “transmitting vehicle”),wherein the motion plan comprises a trajectory of the transmittingvehicle and one or more vehicle descriptors associated with thetransmitting vehicle, parsing the intention message to identify themotion plan for the transmitting vehicle, and controlling the vehiclebased at least in part on the motion plan. In some aspects, the one ormore vehicle descriptors may include a sensor perceptible attribute.

In some aspects, controlling the vehicle based at least in part on themotion plan may include determining an expected region of interest forthe vehicle based at least in part on the motion plan, and applying adetection algorithm to received sensor data at the expected region ofinterest to detect the transmitting vehicle in the received sensor databased at least in part on the sensor perceptible attribute. In someaspects, the method may further include selecting the detectionalgorithm based at least in part on the received motion plan.

In some aspects, controlling the vehicle based at least in part on themotion plan may include correlating vehicle detection sensor data withthe transmitting vehicle based at least in part on the sensorperceptible attribute.

In some aspects, the one or more vehicle descriptors may include avehicle physical capability. In some aspects, controlling the vehiclebased at least in part on the motion plan may include setting a behaviorprediction for the transmitting vehicle based at least in part on themotion plan. Some aspects may further include determining whether abehavior of the transmitting vehicle conforms to the behaviorprediction, and updating the behavior prediction based at least in parton the vehicle physical capability in response to determining that thebehavior of the transmitting vehicle does not conform to the behaviorprediction.

In some aspects, the one or more vehicle descriptors may include avehicle location attribute. In some aspects, controlling the vehiclebased at least in part on the motion plan may include determining aposition of the transmitting vehicle based at least in part on thevehicle location attribute, determining whether a comparison between aposition of the vehicle and the position of the transmitting vehicleindicate an error, and triggering a recalculation of the position of thevehicle in response to determining the comparison between the positionof the vehicle and the position of the transmitting vehicle indicate anerror.

In some aspects, controlling the vehicle based at least in part on themotion plan may include determining whether the motion plan is unsafe,and sending a safety warning to the transmitting vehicle in response todetermining the motion plan is unsafe.

Various aspects for broadcasting a message from a vehicle may includedetermining a motion plan for the vehicle, wherein the motion plancomprises a trajectory of the vehicle and one or more vehicledescriptors of the vehicle, generating an intention message based atleast in part on the determined motion plan, and broadcasting theintention message from the vehicle. In some aspects, the one or morevehicle descriptors may include a sensor perceptible attribute, avehicle physical capability, or a vehicle location attribute.

Further aspects include a vehicle including a processor configured withprocessor-executable instructions to perform operations of any of themethods summarized above. Further aspects include a non-transitoryprocessor-readable storage medium having stored thereonprocessor-executable software instructions configured to cause aprocessor to perform operations of any of the methods summarized above.Further aspects include a processing device for use in a vehicle andconfigured to perform operations of any of the methods summarized above.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and constitutepart of this specification, illustrate exemplary embodiments, andtogether with the general description given above and the detaileddescription given below, serve to explain the features of the variousembodiments.

FIGS. 1A and 1B are component block diagrams illustrating a vehiclesuitable for implementing various embodiments.

FIG. 1C is a component block diagram illustrating components of avehicle suitable for implementing various embodiments.

FIG. 2A is a component block diagram illustrating components of anexample vehicle management system according to various embodiments.

FIG. 2B is a component block diagram illustrating components of anotherexample vehicle management system according to various embodiments.

FIG. 3 is a block diagram illustrating components of an example systemon chip for use in a vehicle that may be configured to broadcast,receive, and/or otherwise use intentions and/or motion plans inaccordance with various embodiments.

FIG. 4 is a process flow diagram illustrating a method of broadcastingan intention message according to various embodiments.

FIG. 5 is a process flow diagram illustrating a method of extracting amotion plan from a broadcast intention message according to variousembodiments.

FIG. 6 is a process flow diagram illustrating a method of using abroadcast motion plan in sensor perception operations according tovarious embodiments.

FIG. 7 is a process flow diagram illustrating a method of using abroadcast motion plan in sensor fusion operations according to variousembodiments.

FIG. 8A is a process flow diagram illustrating a method of using abroadcast motion plan in behavior prediction operations according tovarious embodiments.

FIG. 8B is a process flow diagram illustrating a method of using abroadcast motion plan in behavior prediction operations according tovarious embodiments.

FIG. 9 is a process flow diagram illustrating a method of using abroadcast motion plan in position localization operations according tovarious embodiments.

FIG. 10 is a process flow diagram illustrating a method of using abroadcast motion plan to share burden of safety operations betweenvehicles according to various embodiments.

DETAILED DESCRIPTION

Various aspects will be described in detail with reference to theaccompanying drawings. Wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.References made to particular examples and embodiments are forillustrative purposes and are not intended to limit the scope of thevarious aspects or the claims.

The surface transportation industry has increasingly looked to leveragethe growing capabilities of cellular and wireless communicationtechnologies through the adoption of Intelligent Transportation Systems(ITS) technologies to increase intercommunication and safety for bothdriver-operated vehicles and autonomous vehicles. The cellularvehicle-to-everything (C-V2X) protocol defined by the 3^(rd) GenerationPartnership Project (3GPP) supports ITS technologies and serves as thefoundation for vehicles to communicate directly with the communicationdevices around them.

C-V2X defines two transmission modes that, together, provide a 360°non-line-of-sight awareness and a higher level of predictability forenhanced road safety and autonomous driving. A first transmission modeincludes direct C-V2X, which includes vehicle-to-vehicle (V2V),vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P), andthat provides enhanced communication range and reliability in thededicated ITS 5.9 gigahertz (GHz) spectrum that is independent of acellular network. A second transmission mode includes vehicle-to-networkcommunications (V2N) in mobile broadband systems and technologies, suchas third generation wireless mobile communication technologies (3G)(e.g., global system for mobile communications (GSM) evolution (EDGE)systems, code division multiple access (CDMA) 2000 systems, etc.),fourth generation wireless mobile communication technologies (4G) (e.g.,long term evolution (LTE) systems, LTE-Advanced systems, mobileWorldwide Interoperability for Microwave Access (mobile WiMAX) systems,etc.), fifth generation wireless mobile communication technologies (5G)(e.g., 5G New Radio (5G NR) systems, etc.), etc.

The term “system-on-chip” (SOC) is used herein to refer to a set ofinterconnected electronic circuits typically, but not exclusively,including one or more processors, a memory, and a communicationinterface. The SOC may include a variety of different types ofprocessors and processor cores, such as a general purpose processor, acentral processing unit (CPU), a digital signal processor (DSP), agraphics processing unit (GPU), an accelerated processing unit (APU), asub-system processor, an auxiliary processor, a single-core processor,and a multicore processor. The SOC may further embody other hardware andhardware combinations, such as a field programmable gate array (FPGA), aconfiguration and status register (CSR), an application-specificintegrated circuit (ASIC), other programmable logic device, discretegate logic, transistor logic, registers, performance monitoringhardware, watchdog hardware, counters, and time references. SOCs may beintegrated circuits (ICs) configured such that the components of the ICsreside on the same substrate, such as a single piece of semiconductormaterial (e.g., silicon, etc.).

Various embodiments include methods, vehicles, vehicle managementsystems, and processing devices configured to implement the methods forbroadcasting, receiving, and/or otherwise using intentions and/or motionplans during operation of vehicles, such as autonomous vehicles,semi-autonomous vehicles, driver-operated vehicles, etc.

Autonomous and semi-autonomous vehicles, such as cars and trucks, arebecoming a reality on city streets. Autonomous and semi-autonomousvehicles typically include a plurality of sensors, including cameras,radar, and lidar, that collect information about the environmentsurrounding the vehicle. For example, such collected information mayenable the vehicle to recognize the roadway, identify objects to avoid,and track the movement and future position of other vehicles to enablepartial or fully autonomous navigation. Similarly, non-autonomousvehicles, such as vehicles that cannot operate in an autonomous drivingor semi-autonomous driving mode, may also include a plurality ofsensors, including cameras, radar, and lidar, that collect informationabout the environment surrounding the vehicle. For example, suchcollected information may enable the vehicle to recognize the roadway,identify objects to avoid, and track the movement and future position ofother vehicles to provide warnings to a driver of the vehicle

A problem in autonomous driving of selecting an optimal driving actionin an uncertain environment with uncertain agents can be modeled as apartially-observable Markov decision process (POMDP). In general,solutions to POMDPs are computationally intractable, so much research isdevoted to finding ways to sufficiently approximate the problem suchthat online and real-time solutions are possible. Autonomous vehiclescan simulate possible actions in order to determine the range ofexpected outcomes based on each action. There may be some reward or riskor penalty for each action and testing each possible action that can besearched can allow the autonomous vehicle to select the action with themost reward and/or least likely penalty.

Autonomous driving stacks are predominantly designed as independent,standalone systems. In other words, an autonomous vehicle is tasked withinferring its own belief about the state of the world and its evolutionwithout help from the other agents in the environment. Equipped onlywith its onboard sensors and computing power, the autonomous vehicle'sbelief of the world can be uncertain and there are infinitely manypossibilities for how the world may evolve in time. Thus, the autonomousvehicle is required to search very large spaces of possibilities todecide what action to take next.

Incorporating C-V2X (connected vehicles sharing information) withautonomous driving stacks can significantly reduce the dimensionality ofthe POMDP by increasing the certainty of important pieces ofinformation. Various embodiments provide for broadcasting an intentionmessage from an autonomous (or semi-autonomous) vehicle comprising amotion plan for that vehicle. A motion plan may be an indication of thatvehicle's current position and an indication of how that vehicle expectsits position to change over time. For example, a motion plan mayindicate a position of a vehicle and a trajectory the vehicle willfollow for a period of time. In various embodiments, the indication ofhow that vehicle expects its position to change over time may be anaffirmative indication of a decision already made by that vehicle. Inthat manner, motion plans indicate an expected course of action and maybe distinguished from requests to take an action (or other type ofconfirmation requiring communications) that may be exchanged betweenvehicles. The sharing of a motion plan via broadcasting intentions ofautonomous (or semi-autonomous) vehicles to surrounding vehicles mayprovide a much greater benefit within the POMDP and may reduce the POMDPto a Markov decision process (MDP) that may require far lesscomputational resources to solve.

Various embodiments enable autonomous (or semi-autonomous) vehicles tobroadcast their respective intentions and/or motion plans to surroundingvehicles, such as autonomous vehicles, semi-autonomous vehicles,non-autonomous vehicles, etc. There may be advantages to thecommunication of intentions and/or motion plan information throughoutthe entire autonomous vehicle software stack.

In various embodiments, a motion plan may be a trajectory. An autonomous(or semi-autonomous) vehicle may know its current position and thedirection and/or next action the autonomous (or semi-autonomous) vehiclemay take to arrive at a next position at a next time. For example, theautonomous vehicle may have a trajectory to get to a next point from acurrent point. In various embodiments, a motion plan may include atrajectory and one or more vehicle descriptors associated with thevehicle and/or the vehicle owner and/or operator. For example, vehicledescriptors may include sensor perceptible attributes of the vehicle,such as vehicle color, vehicle license plate number, vehicle size, etc.As a further example, vehicle descriptors may include vehicle physicalcapabilities, such as vehicle type, vehicle turning radius, vehicle topspeed, vehicle maximum acceleration, etc. As a still further example,vehicle descriptors may include vehicle location attributes, such as thevehicle's latitude and longitude, the vehicle's distance from and/ororientation to a known landmark in a coordinate plane, etc. Sharing themotion plan with other vehicles, such as other autonomous vehicles,other semi-autonomous vehicles, other non-autonomous vehicles, etc., mayprovide benefits to the other vehicles receiving the motion plan and/orthe vehicle transmitting the motion plan.

In various embodiments, a vehicle, such as an autonomous vehicle, asemi-autonomous vehicle, a non-autonomous vehicle, etc., may receive anintention message including at least a motion plan. In variousembodiments, the motion plan may be shared among one or more (e.g., all)of the components throughout the autonomous vehicle stack and/or may beshared among one or more (e.g., all) other components of the vehicle.Sharing that motion plan may enable a transmitting vehicle to share itsposition and share how that position is expect to evolve over time forthat sharing vehicle. Sharing a motion plan may enable an autonomous orsemi-autonomous vehicle receiving the motion plan to determine how themotions of the vehicle transmitting the motion plan will affect thevehicle receiving the motion plan.

In various embodiments, an intention message may include an identifierof the vehicle broadcasting the intention message, the current positionof the vehicle broadcasting the intention message, a motion plan of thevehicle broadcasting the intention message, and/or other data related tothe vehicle broadcasting the intention message.

In various embodiments, a motion plan may include a vehicle's positionand an indication of how the position over time to is expected tochange. In some embodiments, a motion plan may include a vehicle'strajectory. In some embodiments, a motion plan may include a vehicle'strajectory and one or more vehicle descriptors associated with thevehicle and/or the vehicle owner and/or operator. In some embodiments, amotion plan may include an indication of an expected next position ofthe reporting vehicle at a certain time. In some embodiments, a motionplan may include a vehicle's motion vector. In some embodiments, amotion plan may include an indication of the coordinate plane used fordetermining the vehicle's indicated position. In various embodiments,the motion plan may describe features of the vehicle transmitting themotion plan, such as its size, orientation, color, vehicle type, etc. Invarious embodiments, the motion plan may indicate the speed of thevehicle transmitting the motion plan, orientation of the vehicletransmitting the motion plan, acceleration of the vehicle transmittingthe motion plan, or any other state information of the vehicletransmitting the motion plan. In various embodiments, the motion planmay indicate future actions (or intentions) of the vehicle transmittingthe motion plan, such as “turning on left blinker in five seconds”,“turning right in two seconds”, “braking in one hundred feet”, or anyother type actions or intentions relevant to driving.

In various embodiments, intention messages may be broadcast from anautonomous (or semi-autonomous) vehicle, such as by C-V2X transmissionmodes. In various embodiments, intention messages may be broadcastperiodically, such as at a set time interval, in response to a change inintention of a broadcasting vehicle, etc.

In various embodiments, a vehicle, such as an autonomous vehicle, asemi-autonomous vehicle, a non-autonomous vehicle, etc., may receive abroadcast intention message and may parse the received intention messageto determine the broadcasting vehicle's identifier, the current positionof the intention message, and the motion plan, and/or any otherinformation indicated within the intention message. In variousembodiments, broadcast intention messages may be received and parsedregardless of the operating mode of the vehicle. For example, broadcastintention messages may be received by an autonomous or semi-autonomousvehicle being actively controlled by a driver at a given time. Invarious embodiments, the broadcasting vehicle's identifier, the currentposition of the intention message, and the motion plan, and/or any otherinformation indicated within the intention message may be provided tovarious hardware and software components of the receiving vehicle. Forexample, the broadcasting vehicle's identifier, the current position ofthe intention message, and the motion plan, and/or any other informationindicated within the intention message may be stored in one or morememory locations on the receiving vehicle, may be sent to one or morelayers of an autonomous vehicle management system, may be sent to one ormore layers of a vehicle management system, may be sent to a vehiclesafety and crash avoidance system, etc. In various embodiments, motionplans may be used by one or more layers of the vehicle management systemto augment various decision making and/or autonomous driving operations.As examples, a received motion plan may be used by the vehiclemanagement system in: sensor fusion processing; behavior prediction;behavioral planning; motion planning; position localization; and/orsharing the burden of safety operations between vehicles.

In various embodiments, a received motion plan broadcast from anothervehicle may be used in sensor perception operations of a vehicle, suchas an autonomous vehicle, a semi-autonomous vehicle, a non-autonomousvehicle, etc. Sensor perception may include operations to control wherea sensor looks to confirm a detection of an object, such as anothervehicle. In various embodiments, a motion plan for another vehicle mayenable a region of interest for perceiving that vehicle in sensor datafor the receiving vehicle to be narrowed to the perceptual space definedby the motion plan broadcasted by the autonomous vehicle. Sensorperceptual data on where the system would expect that car to be mayconfirm or deny whether the vehicle receiving the motion plan (e.g., anautonomous vehicle, a semi-autonomous vehicle, a non-autonomous vehicle,etc.) is handling detection of other vehicles correctly. In perceptionsystems, vehicles must be picked out of potentially terabytes of datacoming into the vehicle from many different cameras, many differentradars, and many other different sensors. The received data from thesesensor needs to be processed by the perception layers in fractions ofseconds to identify objects in the data and get that processed sensordata to other layers in the vehicle management system. Being able tofocus on a specific region of interest in the data in the space aroundthe vehicle because an object, such as the motion plan broadcastingvehicle, is expected in that region of interest may increase the speedof detection of that vehicle when compared with analyzing the data as awhole. Additionally, when a motion plan indicates vehicle descriptorsthat may be sensor perceptible attributes, being able to focus thesensor toward a specific sensor perceptible attribute, such as vehiclecolor, vehicle size, etc., may increase the speed of detection of thatvehicle when compared with analyzing the data as a whole. In variousembodiments, the motion plan may be used to determine the region ofinterest to apply to the raw sensor data to perceive a vehicle in thatraw sensor data. In some embodiments, the motion plan may be used toselect and/or modify the algorithm used to perceive vehicles in the rawsensor data. For example, a different algorithm may be used when themotion plan broadcasting vehicle is expected to be head-on to thereceiving vehicle than when the motion plan broadcasting vehicle isexpected to be perpendicular to the receiving vehicle. As a furtherexample, a different detection threshold may be used depending onwhether a motion plan indicates the broadcasting vehicle is expected tobe in a given space. As a specific example, without a motion plan areceiving vehicle may only report detections that pass with a 90%confidence, while with a motion plan the receiving vehicle may reportdetections that pass a 50% confidence in a region of interest associatedwith the motion plan. In some embodiments, the motion plan may be usedto confirm whether or not a specific vehicle is perceived in the rawsensor data. For example, the detection of a sensor perceptibleattribute of a vehicle included in a motion plan as a vehicle descriptor(e.g., the vehicle's color, vehicle's license plate number, etc.) in theraw sensor data may confirm that the vehicle transmitting the motionplan was the actual vehicle perceived in a region of interest by anothervehicle in the vicinity.

In various embodiments, a received motion plan broadcast from anothervehicle may be used in sensor fusion operations of a vehicle, such as anautonomous vehicle, a semi-autonomous vehicle, a non-autonomous vehicle,etc. Sensor fusion operations may be operations to combine and associatesensor data and associate that sensor data with a tracked object. Amotion plan may improve the performance of data association and trackingoperations of a sensor fusion layer of a vehicle management system. Thevehicle management system may use the motion plan to determine how tofuse all the raw detections of other vehicles in an environmenttogether. For example, based on the motion plan broadcast for a vehicle,the receiving vehicle may be enabled to determine a radar detection, aLIDAR detection, and a camera detection are actually all the samevehicle because the detections all correlate to the received motion planfor that vehicle rather than initially treating the three detections asseparate vehicles. As a specific example, different sensors of thereceiving vehicle positively identifying one or more sensor perceptibleattributes of a vehicle included in a motion plan may confirm that thesensors are detecting the same vehicle that sent the motion plan.Additionally, the motion plan may enable the receiving vehicle todetermine that those vehicle detections will evolve together. Theability to compare vehicle detections to motion plans may enable outliermeasurements to be discarded. For example, a tracked vehicle's motionplan indicating it intends to stay in a current lane may enable adetection not corresponding to that lane to be associated with a newobject rather than the previously tracked vehicle. The presence of amotion plan may reduce uncertainty in the sensor fusion operations. Thenoisy detections from the perception layer may be compared with theunderlying trajectories and/or vehicle descriptors in a motion plan togive an improved certainty to sensor fusion operations.

In various embodiments, a received motion plan broadcast from anothervehicle may be used in behavior prediction, behavioral planning, and/ormotion planning operations of a vehicle, such as an autonomous vehicle,a semi-autonomous vehicle, a non-autonomous vehicle, etc. The broadcastof a motion plan by a reporting vehicle may enable a vehicle receivingthat motion plan to predict the behavior of that vehicle with adefinable certainty. The benefit of a motion plan and a position may bethat the motion plan may be treated as the predicted behavior of thatvehicle. This certainty in behavior prediction may reduce thedimensionality of the POMDP by knowing the behavioral/trajectoryprediction exactly for surrounding cars that broadcast their respectivemotion plans. Sharing the motion plan may result in behavioralpredictions with small or no uncertainty. Additionally, knowing theintended motion of a vehicle may eliminate the need to estimate thatvehicle's motion, thereby reducing the computational resourcesassociated with behavior prediction.

In various embodiments, receiving a motion plan broadcast by a vehiclereduces the behavioral planning searches over a smaller space ofpossibilities. Given the perfect (or near perfect) knowledge of thestate and actions of other vehicles provided by those vehicles'broadcast motion plans, finding optimal actions for the receivingvehicle collapses from a POMDP to an MDP analysis in which there are ahost of solutions suitable for online and real-time operations. As theuncertain number of infinite actions of the broadcasting vehicle arereduced to a finite number of intended actions by receiving the motionplan for that broadcasting vehicle, the behavioral planning layer of thevehicle stack may develop high level driving goals for the receivingvehicle. Additionally, a motion plan including vehicle descriptors thatreflect vehicle physical capabilities of the vehicle transmitting themotion plan may reduce the space of possibilities for behavioralplanning searches performed by a vehicle management system of areceiving vehicle by limiting the possible behaviors of the transmittingvehicle to be assessed by a vehicle behavior model to those within thetransmitting vehicle's capabilities. For example, a type of vehicleindicated in a motion plan may be used by a receiving vehicle managementsystem to constrain a maximum speed or acceleration of the vehicle in avehicle behavior model based on the maximum speed or accelerationassociated with that vehicle type. As another example, a turning radiusof the vehicle indicated in a motion plan may be used by a receivingvehicle management system to constrain the potential turning pathsmodeled for the vehicle transmitting the motion plan to within theindicated turning radius.

In various embodiments, a vehicle management system receiving a motionplan broadcast by a vehicle including vehicle descriptors that reflectvehicle physical capabilities of the transmitting vehicle may use thatinformation to reduce the space of possibilities for behavioral planningsearches within a vehicle behavior model, such as after observing thevehicle deviating from a behavior prediction. In response to determiningthat a vehicle's observed behavior does not conform to a behaviorprediction made by the vehicle management system, the behaviorprediction for that vehicle may be updated based at least in part onvehicle capabilities determined from the received motion plan. Forexample, the vehicle management system may use the other vehicle'sphysical capabilities to collapse the possible future behaviors of theother vehicle determined by a vehicle behavior model from a POMDP to anMDP analysis in which there are a host of solutions suitable for onlineand real-time operations. For example, a type of vehicle indicated in areceived motion plan may be used by the receiving vehicle managementsystem to constrain a maximum speed or acceleration of the other vehicleused in updating the behavior prediction made by the vehicle behaviormodel based on the maximum speed or acceleration associated with thatthe type of the other vehicle. As another example, a turning radius ofthe other vehicle indicated in a motion plan may be used by thereceiving vehicle management system to constrain the vehicle behaviormodel for potential turning paths of the vehicle transmitting the motionplan to within the indicated turning radius and use the vehicle behaviormodel updated in this manner to generate an updated other vehiclebehavior prediction.

In various embodiments, a behavioral planning layer may provide ahigh-level driving goal to a motion planning layer. The motion planninglayer may be responsible for actually planning the trajectory to executethat high-level maneuver and the motion planning layer may beresponsible for ensuring the safety of executing that trajectory.Receiving a motion plan broadcast by another vehicle in the environmentmay enable the motion planning layer to perform fewer collision checksas a result of a prediction of the motion of that broadcasting vehiclebeing known with certainty. Collision checking is often a bottleneck offast motion planning, so fewer checks may greatly speed up the overallautonomous driving algorithm. In various embodiments, a behavioralplanning layer may provide high level driving goals and/or behaviormodels for other vehicles to a vehicle safety and crash avoidancesystem. The vehicle safety and crash avoidance system may use the highlevel driving goals and/or behavior models for other vehicles to performsafety checks, such as collision checks, etc., while the vehicle isdriving to avoid crashes.

In various embodiments, a received motion plan broadcast from anothervehicle may be used in position localization operations of a vehicle,such as an autonomous vehicle, semi-autonomous vehicle, non-autonomousvehicle, etc. In various embodiments, a localization layer of a vehiclemanagement system may leverage the observations of the broadcastingvehicle to improve the position estimate of the receiving vehicle. Insome embodiments, the localization layer (or positioning layer) mayutilize the sensor fusion output describing the state of other vehiclesand compare it to where these other vehicles are expected to be based ontheir broadcasted intentions in their respective motion plans. This maybe similar to how map fusion is often done to augment positioning bycomparing observation of lanes and landmarks to their expected positionsbased on an a priori known map. In this manner, the own vehiclelocalization process may leverage the observations of other vehiclesusing their respective motion plans. Thus, the receiving vehicle maybetter localize itself based on the broadcast motion plans ofsurrounding autonomous vehicles.

In various embodiments, localization may leverage a high-quality map ina global and/or local coordinate plan including known positions oflandmarks in that map, such as lane markers, road signs, etc. In variousembodiments, intention messages and/or motion plans broadcast byvehicles may indicate those vehicles distance from, and/or orientationto, a known landmark in a coordinate plane, such as a local and/orglobal coordinate plane. For example, a motion plan may indicate thetransmitting vehicle's distance from, and/or orientation to, a knownlandmark in a coordinate plane, such as a local and/or global coordinateplane, such as in a vehicle location attribute type vehicle descriptor.A vehicle, such as an autonomous vehicle, semi-autonomous vehicle,non-autonomous vehicle, etc., receiving the indications from thebroadcasting vehicles may compare its observations of those broadcastingvehicles and their position relative to the known landmark to thedistance from the known landmark in the intention message and/or motionplan to assist in localization process performed on the receivingvehicle. For example, the receiving vehicle may compare its ownobservations to those in the received motion plans to determine whetherthere is an error or offset between its observations and those in themotion plans. An error or offset between its observations and those inthe received motion plans may indicate to the receiving vehicle that ithas localized its respective current position incorrectly. In response,the receiving vehicle may trigger a recalculation of its position. Invarious embodiments, the receiving vehicle may convert the observationsin an intention message and/or motion plan from one coordinate plane toanother coordinate plane. For example, the vehicle may convert theobservations of a broadcasting vehicle from a global coordinate plane(e.g., latitude & longitude) to a local (e.g., street map-centric)coordinate plane. As a specific example, two different autonomousvehicles may both broadcast their respective motion plans and thosemotion plans may be received by the receiving vehicle. Observations of alandmark in those two motion plans may match, but may be different fromthe observation of the receiving vehicle of that same landmark. Theagreement of two observations in the different motion plans and theirdifference from the receiving vehicle's observations may indicate thereceiving vehicle localized its position wrong. In response, thereceiving vehicle may recalculate its position.

In various embodiments, a received motion plan broadcast from anothervehicle may be used to share the burden of safety operations betweenvehicles, such as autonomous vehicles, semi-autonomous vehicles, etc. Invarious embodiments, broadcasting motion plans also allows forextensions like sharing the burden of safety between the equippedvehicles. A vehicle can verify that received motion plans are safewithin its own belief of the environment, and issue warnings back to thebroadcasting vehicle if safety appears compromised by the receivedmotion plan. For example, the receiving vehicle may detect an objectthat will cause a motion plan of a broadcasting vehicle to be unsafeeven though the broadcasting vehicle did not yet sense or otherwiseobserve that object. The receiving vehicle may determine the motion planis unsafe and may indicate a warning to the broadcasting vehicle. Thewarning may include the observation of the object causing the motionplan to be unsafe.

Another advantage of broadcasting intentions is that nothing is impactedin the vehicle management system if no other vehicles in the vicinitybroadcast intention messages. If no intention messages are received, thevehicle management system may revert to making the vehicle responsiblefor the full inference of the state of the world.

Various embodiments may be implemented within a variety of vehicles, anexample vehicle 100 of which is illustrated in FIGS. 1A and 1B. Withreference to FIGS. 1A and 1B, a vehicle 100 may include a control unit140 and a plurality of sensors 102-138, including satellitegeopositioning system receivers 108, occupancy sensors 112, 116, 118,126, 128, tire pressure sensors 114, 120, cameras 122, 136, microphones124, 134, impact sensors 130, radar 132, and lidar 138. The plurality ofsensors 102-138, disposed in or on the vehicle, may be used for variouspurposes, such as autonomous and semi-autonomous navigation and control,crash avoidance, position determination, etc., as well to provide sensordata regarding objects and people in or on the vehicle 100. The sensors102-138 may include one or more of a wide variety of sensors capable ofdetecting a variety of information useful for navigation and collisionavoidance. Each of the sensors 102-138 may be in wired or wirelesscommunication with a control unit 140, as well as with each other. Inparticular, the sensors may include one or more cameras 122, 136 orother optical sensors or photo optic sensors. The sensors may furtherinclude other types of object detection and ranging sensors, such asradar 132, lidar 138, IR sensors, and ultrasonic sensors. The sensorsmay further include tire pressure sensors 114, 120, humidity sensors,temperature sensors, satellite geopositioning sensors 108,accelerometers, vibration sensors, gyroscopes, gravimeters, impactsensors 130, force meters, stress meters, strain sensors, fluid sensors,chemical sensors, gas content analyzers, pH sensors, radiation sensors,Geiger counters, neutron detectors, biological material sensors,microphones 124, 134, occupancy sensors 112, 116, 118, 126, 128,proximity sensors, and other sensors.

The vehicle control unit 140 may be configured with processor-executableinstructions to perform various embodiments using information receivedfrom various sensors, particularly the cameras 122, 136. In someembodiments, the control unit 140 may supplement the processing ofcamera images using distance and relative position (e.g., relativebearing angle) that may be obtained from radar 132 and/or lidar 138sensors. The control unit 140 may further be configured to controlsteering, breaking and speed of the vehicle 100 when operating in anautonomous or semi-autonomous mode using information regarding othervehicles determined using various embodiments.

FIG. 1C is a component block diagram illustrating a system 150 ofcomponents and support systems suitable for implementing variousembodiments. With reference to FIGS. 1A, 1B, and 1C, a vehicle 100 mayinclude a control unit 140, which may include various circuits anddevices used to control the operation of the vehicle 100. In the exampleillustrated in FIG. 1C, the control unit 140 includes a processor 164,memory 166, an input module 168, an output module 170 and a radio module172. The control unit 140 may be coupled to and configured to controldrive control components 154, navigation components 156, and one or moresensors 158 of the vehicle 100.

As used herein, the terms “component,” “system,” “unit,” “module,” andthe like include a computer-related entity, such as, but not limited to,hardware, firmware, a combination of hardware and software, software, orsoftware in execution, which are configured to perform particularoperations or functions. For example, a component may be, but is notlimited to, a process running on a processor, a processor, an object, anexecutable, a thread of execution, a program, and/or a computer. By wayof illustration, both an application running on a communication deviceand the communication device may be referred to as a component. One ormore components may reside within a process and/or thread of executionand a component may be localized on one processor or core and/ordistributed between two or more processors or cores. In addition, thesecomponents may execute from various non-transitory computer readablemedia having various instructions and/or data structures stored thereon.Components may communicate by way of local and/or remote processes,function or procedure calls, electronic signals, data packets, memoryread/writes, and other known computer, processor, and/or process relatedcommunication methodologies.

The control unit 140 may include a processor 164 that may be configuredwith processor-executable instructions to control maneuvering,navigation, and/or other operations of the vehicle 100, includingoperations of various embodiments. The processor 164 may be coupled tothe memory 166. The control unit 162 may include the input module 168,the output module 170, and the radio module 172.

The radio module 172 may be configured for wireless communication. Theradio module 172 may exchange signals 182 (e.g., command signals forcontrolling maneuvering, signals from navigation facilities, etc.) witha network transceiver 180, and may provide the signals 182 to theprocessor 164 and/or the navigation unit 156. In some embodiments, theradio module 172 may enable the vehicle 100 to communicate with awireless communication device 190 through a wireless communication link192. The wireless communication link 192 may be a bidirectional orunidirectional communication link, and may use one or more communicationprotocols.

The input module 168 may receive sensor data from one or more vehiclesensors 158 as well as electronic signals from other components,including the drive control components 154 and the navigation components156. The output module 170 may be used to communicate with or activatevarious components of the vehicle 100, including the drive controlcomponents 154, the navigation components 156, and the sensor(s) 158.

The control unit 140 may be coupled to the drive control components 154to control physical elements of the vehicle 100 related to maneuveringand navigation of the vehicle, such as the engine, motors, throttles,steering elements, flight control elements, braking or decelerationelements, and the like. The drive control components 154 may alsoinclude components that control other devices of the vehicle, includingenvironmental controls (e.g., air conditioning and heating), externaland/or interior lighting, interior and/or exterior informationaldisplays (which may include a display screen or other devices to displayinformation), safety devices (e.g., haptic devices, audible alarms,etc.), and other similar devices.

The control unit 140 may be coupled to the navigation components 156,and may receive data from the navigation components 156 and beconfigured to use such data to determine the present position andorientation of the vehicle 100, as well as an appropriate course towarda destination. In various embodiments, the navigation components 156 mayinclude or be coupled to a global navigation satellite system (GNSS)receiver system (e.g., one or more Global Positioning System (GPS)receivers) enabling the vehicle 100 to determine its current positionusing GNSS signals. Alternatively, or in addition, the navigationcomponents 156 may include radio navigation receivers for receivingnavigation beacons or other signals from radio nodes, such as Wi-Fiaccess points, cellular network sites, radio station, remote computingdevices, other vehicles, etc. Through control of the drive controlelements 154, the processor 164 may control the vehicle 100 to navigateand maneuver. The processor 164 and/or the navigation components 156 maybe configured to communicate with a server 184 on a network 186 (e.g.,the Internet) using a wireless connection 182 with a cellular datanetwork 180 to receive commands to control maneuvering, receive datauseful in navigation, provide real-time position reports, and assessother data.

The control unit 162 may be coupled to one or more sensors 158. Thesensor(s) 158 may include the sensors 102-138 as described, and may theconfigured to provide a variety of data to the processor 164.

While the control unit 140 is described as including separatecomponents, in some embodiments some or all of the components (e.g., theprocessor 164, the memory 166, the input module 168, the output module170, and the radio module 172) may be integrated in a single device ormodule, such as a system-on-chip (SOC) processing device. Such an SOCprocessing device may be configured for use in vehicles and beconfigured, such as with processor-executable instructions executing inthe processor 164, to perform operations of various embodiments wheninstalled into a vehicle.

FIG. 2A illustrates an example of subsystems, computational elements,computing devices or units within a vehicle management system 200, whichmay be utilized within a vehicle 100. With reference to FIGS. 1A-2A, insome embodiments, the various computational elements, computing devicesor units within vehicle management system 200 may be implemented withina system of interconnected computing devices (i.e., subsystems), thatcommunicate data and commands to each other (e.g., indicated by thearrows in FIG. 2A). In other embodiments, the various computationalelements, computing devices or units within vehicle management system200 may be implemented within a single computing device, such asseparate threads, processes, algorithms or computational elements.Therefore, each subsystem/computational element illustrated in FIG. 2Ais also generally referred to herein as “layer” within a computational“stack” that constitutes the vehicle management system 200. However, theuse of the terms layer and stack in describing various embodiments arenot intended to imply or require that the corresponding functionality isimplemented within a single autonomous (or semi-autonomous) vehiclemanagement system computing device, although that is a potentialimplementation embodiment. Rather the use of the term “layer” isintended to encompass subsystems with independent processors,computational elements (e.g., threads, algorithms, subroutines, etc.)running in one or more computing devices, and combinations of subsystemsand computational elements.

In various embodiments, the vehicle management system stack 200 mayinclude a radar perception layer 202, a camera perception layer 204, apositioning engine layer 206, a map fusion and arbitration layer 208, aroute planning layer 210, sensor fusion and road world model (RWM)management layer 212, motion planning and control layer 214, andbehavioral planning and prediction layer 216. The layers 202-216 aremerely examples of some layers in one example configuration of thevehicle management system stack 200. In other configurations consistentwith various embodiments, other layers may be included, such asadditional layers for other perception sensors (e.g., LIDAR perceptionlayer, etc.), additional layers for planning and/or control, additionallayers for modeling, etc., and/or certain of the layers 202-216 may beexcluded from the vehicle management system stack 200. Each of thelayers 202-216 may exchange data, computational results and commands asillustrated by the arrows in FIG. 2A. Further, the vehicle managementsystem stack 200 may receive and process data from sensors (e.g., radar,lidar, cameras, inertial measurement units (IMU) etc.), navigationsystems (e.g., GPS receivers, IMUs, etc.), vehicle networks (e.g.,Controller Area Network (CAN) bus), and databases in memory (e.g.,digital map data). The vehicle management system stack 200 may outputvehicle control commands or signals to the drive by wire (DBW)system/control unit 220, which is a system, subsystem or computingdevice that interfaces directly with vehicle steering, throttle andbrake controls. The configuration of the vehicle management system stack200 and DBW system/control unit 220 illustrated in FIG. 2A is merely anexample configuration and other configurations of a vehicle managementsystem and other vehicle components may be used in the variousembodiments. As an example, the configuration of the vehicle managementsystem stack 200 and DBW system/control unit 220 illustrated in FIG. 2Amay be used in a vehicle configured for autonomous or semi-autonomousoperation while a different configuration may be used in anon-autonomous vehicle.

The radar perception layer 202 may receive data from one or moredetection and ranging sensors, such as radar (e.g., 132) and/or lidar(e.g., 138), and process the data to recognize and determine locationsof other vehicles and objects within a vicinity of the vehicle 100. Theradar perception layer 202 may include use of neural network processingand artificial intelligence methods to recognize objects and vehicles,and pass such information on to the sensor fusion and RWM managementlayer 212.

The camera perception layer 204 may receive data from one or morecameras, such as cameras (e.g., 122, 136), and process the data torecognize and determine locations of other vehicles and objects within avicinity of the vehicle 100. The camera perception layer 204 may includeuse of neural network processing and artificial intelligence methods torecognize objects and vehicles, and pass such information on to thesensor fusion and RWM management layer 212.

The positioning engine layer 206 may receive data from various sensorsand process the data to determine a position of the vehicle 100. Thevarious sensors may include, but is not limited to, GPS sensor, an IMU,and/or other sensors connected via a CAN bus. The positioning enginelayer 206 may also utilize inputs from one or more cameras, such ascameras (e.g., 122, 136) and/or any other available sensor, such asradars, LIDARs, etc.

The map fusion and arbitration layer 208 may access data within a highdefinition (HD) map database and receive output received from thepositioning engine layer 206 and process the data to further determinethe position of the vehicle 100 within the map, such as location withina lane of traffic, position within a street map, etc. The HD mapdatabase may be stored in a memory (e.g., memory 166). For example, themap fusion and arbitration layer 208 may convert latitude and longitudeinformation from GPS into locations within a surface map of roadscontained in the HD map database. GPS position fixes include errors, sothe map fusion and arbitration layer 208 may function to determine abest guess location of the vehicle within a roadway based upon anarbitration between the GPS coordinates and the HD map data. Forexample, while GPS coordinates may place the vehicle near the middle ofa two-lane road in the HD map, the map fusion and arbitration layer 208may determine from the direction of travel that the vehicle is mostlikely aligned with the travel lane consistent with the direction oftravel. The map fusion and arbitration layer 208 may pass map-basedlocation information to the sensor fusion and RWM management layer 212.

The route planning layer 210 may utilize the HD map, as well as inputsfrom an operator or dispatcher to plan a route to be followed by thevehicle 100 to a particular destination. The route planning layer 210may pass map-based location information to the sensor fusion and RWMmanagement layer 212. However, the use of a prior map by other layers,such as the sensor fusion and RWM management layer 212, etc., is notrequired. For example, other stacks may operate and/or control thevehicle based on perceptual data alone without a provided map,constructing lanes, boundaries, and the notion of a local map asperceptual data is received.

The sensor fusion and RWM management layer 212 may receive data andoutputs produced by the radar perception layer 202, camera perceptionlayer 204, map fusion and arbitration layer 208, and route planninglayer 210, and use some or all of such inputs to estimate or refine thelocation and state of the vehicle 100 in relation to the road, othervehicles on the road, and other objects within a vicinity of the vehicle100. For example, the sensor fusion and RWM management layer 212 maycombine imagery data from the camera perception layer 204 witharbitrated map location information from the map fusion and arbitrationlayer 208 to refine the determined position of the vehicle within a laneof traffic. As another example, the sensor fusion and RWM managementlayer 212 may combine object recognition and imagery data from thecamera perception layer 204 with object detection and ranging data fromthe radar perception layer 202 to determine and refine the relativeposition of other vehicles and objects in the vicinity of the vehicle.As another example, the sensor fusion and RWM management layer 212 mayreceive information from vehicle-to-vehicle (V2V) communications (suchas via the CAN bus) regarding other vehicle positions and directions oftravel, and combine that information with information from the radarperception layer 202 and the camera perception layer 204 to refine thelocations and motions of other vehicles. The sensor fusion and RWMmanagement layer 212 may output refined location and state informationof the vehicle 100, as well as refined location and state information ofother vehicles and objects in the vicinity of the vehicle, to the motionplanning and control layer 214 and/or the behavior planning andprediction layer 216.

As a further example, the sensor fusion and RWM management layer 212 mayuse dynamic traffic control instructions directing the vehicle 100 tochange speed, lane, direction of travel, or other navigationalelement(s), and combine that information with other received informationto determine refined location and state information. The sensor fusionand RWM management layer 212 may output the refined location and stateinformation of the vehicle 100, as well as refined location and stateinformation of other vehicles and objects in the vicinity of the vehicle100, to the motion planning and control layer 214, the behavior planningand prediction layer 216 and/or devices remote from the vehicle 100,such as a data server, other vehicles, etc., via wirelesscommunications, such as through C-V2X connections, other wirelessconnections, etc.

As a still further example, the sensor fusion and RWM management layer212 may monitor perception data from various sensors, such as perceptiondata from a radar perception layer 202, camera perception layer 204,other perception layer, etc., and/or data from one or more sensorsthemselves to analyze conditions in the vehicle sensor data. The sensorfusion and RWM management layer 212 may be configured to detectconditions in the sensor data, such as sensor measurements being at,above, or below a threshold, certain types of sensor measurementsoccurring, etc., and may output the sensor data as part of the refinedlocation and state information of the vehicle 100 provided to thebehavior planning and prediction layer 216 and/or devices remote fromthe vehicle 100, such as a data server, other vehicles, etc., viawireless communications, such as through C-V2X connections, otherwireless connections, etc.

The refined location and state information may include vehicledescriptors associated with the vehicle and the vehicle owner and/oroperator, such as: vehicle specifications (e.g., size, weight, color, onboard sensor types, etc.); vehicle position, speed, acceleration,direction of travel, attitude, orientation, destination, fuel/powerlevel(s), and other state information; vehicle emergency status (e.g.,is the vehicle an emergency vehicle or private individual in anemergency); vehicle restrictions (e.g., heavy/wide load, turningrestrictions, high occupancy vehicle (HOV) authorization, etc.);capabilities (e.g., all-wheel drive, four-wheel drive, snow tires,chains, connection types supported, on board sensor operating statuses,on board sensor resolution levels, etc.) of the vehicle; equipmentproblems (e.g., low tire pressure, weak breaks, sensor outages, etc.);owner/operator travel preferences (e.g., preferred lane, roads, routes,and/or destinations, preference to avoid tolls or highways, preferencefor the fastest route, etc.); permissions to provide sensor data to adata agency server (e.g., 184); and/or owner/operator identificationinformation.

The behavioral planning and prediction layer 216 of the autonomousvehicle system stack 200 may use the refined location and stateinformation of the vehicle 100 and location and state information ofother vehicles and objects output from the sensor fusion and RWMmanagement layer 212 to predict future behaviors of other vehiclesand/or objects. For example, the behavioral planning and predictionlayer 216 may use such information to predict future relative positionsof other vehicles in the vicinity of the vehicle based on own vehicleposition and velocity and other vehicle positions and velocity. Suchpredictions may take into account information from the HD map and routeplanning to anticipate changes in relative vehicle positions as host andother vehicles follow the roadway. The behavioral planning andprediction layer 216 may output other vehicle and object behavior andlocation predictions to the motion planning and control layer 214.Additionally, the behavior planning and prediction layer 216 may useobject behavior in combination with location predictions to plan andgenerate control signals for controlling the motion of the vehicle 100.For example, based on route planning information, refined location inthe roadway information, and relative locations and motions of othervehicles, the behavior planning and prediction layer 216 may determinethat the vehicle 100 needs to change lanes and accelerate, such as tomaintain or achieve minimum spacing from other vehicles, and/or preparefor a turn or exit. As a result, the behavior planning and predictionlayer 216 may calculate or otherwise determine a steering angle for thewheels and a change to the throttle setting to be commanded to themotion planning and control layer 214 and DBW system/control unit 220along with such various parameters necessary to effectuate such a lanechange and acceleration. One such parameter may be a computed steeringwheel command angle.

The motion planning and control layer 214 may receive data andinformation outputs from the sensor fusion and RWM management layer 212and other vehicle and object behavior as well as location predictionsfrom the behavior planning and prediction layer 216, and use thisinformation to plan and generate control signals for controlling themotion of the vehicle 100 and to verify that such control signals meetsafety requirements for the vehicle 100. For example, based on routeplanning information, refined location in the roadway information, andrelative locations and motions of other vehicles, the motion planningand control layer 214 may verify and pass various control commands orinstructions to the DBW system/control unit 220.

The DBW system/control unit 220 may receive the commands or instructionsfrom the motion planning and control layer 214 and translate suchinformation into mechanical control signals for controlling wheel angle,brake and throttle of the vehicle 100. For example, DBW system/controlunit 220 may respond to the computed steering wheel command angle bysending corresponding control signals to the steering wheel controller.

In various embodiments, the vehicle management system stack 200 mayinclude functionality that performs safety checks or oversight ofvarious commands, planning or other decisions of various layers thatcould impact vehicle and occupant safety. Such safety check or oversightfunctionality may be implemented within a dedicated layer or distributedamong various layers and included as part of the functionality. In someembodiments, a variety of safety parameters may be stored in memory andthe safety checks or oversight functionality may compare a determinedvalue (e.g., relative spacing to a nearby vehicle, distance from theroadway centerline, etc.) to corresponding safety parameter(s), andissue a warning or command if the safety parameter is or will beviolated. For example, a safety or oversight function in the behaviorplanning and prediction layer 216 (or in a separate layer) may determinethe current or future separate distance between another vehicle (asrefined by the sensor fusion and RWM management layer 212) and thevehicle (e.g., based on the world model refined by the sensor fusion andRWM management layer 212), compare that separation distance to a safeseparation distance parameter stored in memory, and issue instructionsto the motion planning and control layer 214 to speed up, slow down orturn if the current or predicted separation distance violates the safeseparation distance parameter. As another example, safety or oversightfunctionality in the motion planning and control layer 214 (or aseparate layer) may compare a determined or commanded steering wheelcommand angle to a safe wheel angle limit or parameter, and issue anoverride command and/or alarm in response to the commanded angleexceeding the safe wheel angle limit.

Some safety parameters stored in memory may be static (i.e., unchangingover time), such as maximum vehicle speed. Other safety parametersstored in memory may be dynamic in that the parameters are determined orupdated continuously or periodically based on vehicle state informationand/or environmental conditions. Non-limiting examples of safetyparameters include maximum safe speed, maximum brake pressure, maximumacceleration, and the safe wheel angle limit, all of which may be afunction of roadway and weather conditions.

FIG. 2B illustrates an example of subsystems, computational elements,computing devices or units within a vehicle management system 250, whichmay be utilized within a vehicle 100. With reference to FIGS. 1A-2B, insome embodiments, the layers 202, 204, 206, 208, 210, 212, and 216 ofthe vehicle management system stack 200 may be similar to thosedescribed with reference to FIG. 2A and the vehicle management systemstack 250 may operate similar to the vehicle management system stack200, except that the vehicle management system stack 250 may passvarious data or instructions to a vehicle safety and crash avoidancesystem 252 rather than the DBW system/control unit 220. For example, theconfiguration of the vehicle management system stack 250 and the vehiclesafety and crash avoidance system 252 illustrated in FIG. 2B may be usedin a non-autonomous vehicle.

In various embodiments, the behavioral planning and prediction layer 216and/or sensor fusion and RWM management layer 212 may output data to thevehicle safety and crash avoidance system 252. For example, the sensorfusion and RWM management layer 212 may output sensor data as part ofrefined location and state information of the vehicle 100 provided tothe vehicle safety and crash avoidance system 252. The vehicle safetyand crash avoidance system 252 may use the refined location and stateinformation of the vehicle 100 to make safety determinations relative tothe vehicle 100 and/or occupants of the vehicle 100. As another example,the behavioral planning and prediction layer 216 may output behaviormodels and/or predictions related to the motion of other vehicles to thevehicle safety and crash avoidance system 252. The vehicle safety andcrash avoidance system 252 may use the behavior models and/orpredictions related to the motion of other vehicles to make safetydeterminations relative to the vehicle 100 and/or occupants of thevehicle 100.

In various embodiments, the vehicle safety and crash avoidance system252 may include functionality that performs safety checks or oversightof various commands, planning, or other decisions of various layers, aswell as human driver actions, that could impact vehicle and occupantsafety. In some embodiments, a variety of safety parameters may bestored in memory and the vehicle safety and crash avoidance system 252may compare a determined value (e.g., relative spacing to a nearbyvehicle, distance from the roadway centerline, etc.) to correspondingsafety parameter(s), and issue a warning or command if the safetyparameter is or will be violated. For example, a vehicle safety andcrash avoidance system 252 may determine the current or future separatedistance between another vehicle (as refined by the sensor fusion andRWM management layer 212) and the vehicle (e.g., based on the worldmodel refined by the sensor fusion and RWM management layer 212),compare that separation distance to a safe separation distance parameterstored in memory, and issue instructions to a driver to speed up, slowdown or turn if the current or predicted separation distance violatesthe safe separation distance parameter. As another example, a vehiclesafety and crash avoidance system 252 may compare a human driver'schange in steering wheel angle to a safe wheel angle limit or parameter,and issue an override command and/or alarm in response to the steeringwheel angle exceeding the safe wheel angle limit.

FIG. 3 illustrates an example system-on-chip (SOC) architecture of aprocessing device SOC 300 suitable for implementing various embodimentsin vehicles. With reference to FIGS. 1A-3, the processing device SOC 300may include a number of heterogeneous processors, such as a digitalsignal processor (DSP) 303, a modem processor 304, an image and objectrecognition processor 306, a mobile display processor 307, anapplications processor 308, and a resource and power management (RPM)processor 317. The processing device SOC 300 may also include one ormore coprocessors 310 (e.g., vector co-processor) connected to one ormore of the heterogeneous processors 303, 304, 306, 307, 308, 317. Eachof the processors may include one or more cores, and anindependent/internal clock. Each processor/core may perform operationsindependent of the other processors/cores. For example, the processingdevice SOC 300 may include a processor that executes a first type ofoperating system (e.g., FreeBSD, LINUX, OS X, etc.) and a processor thatexecutes a second type of operating system (e.g., Microsoft Windows). Insome embodiments, the applications processor 308 may be the SOC's 300main processor, central processing unit (CPU), microprocessor unit(MPU), arithmetic logic unit (ALU), etc. The graphics processor 306 maybe graphics processing unit (GPU).

The processing device SOC 300 may include analog circuitry and customcircuitry 314 for managing sensor data, analog-to-digital conversions,wireless data transmissions, and for performing other specializedoperations, such as processing encoded audio and video signals forrendering in a web browser. The processing device SOC 300 may furtherinclude system components and resources 316, such as voltage regulators,oscillators, phase-locked loops, peripheral bridges, data controllers,memory controllers, system controllers, access ports, timers, and othersimilar components used to support the processors and software clients(e.g., a web browser) running on a computing device.

The processing device SOC 300 also include specialized circuitry forcamera actuation and management (CAM) 305 that includes, provides,controls and/or manages the operations of one or more cameras 122, 136(e.g., a primary camera, webcam, 3D camera, etc.), the video displaydata from camera firmware, image processing, video preprocessing, videofront-end (VFE), in-line JPEG, high definition video codec, etc. The CAM305 may be an independent processing unit and/or include an independentor internal clock.

In some embodiments, the image and object recognition processor 306 maybe configured with processor-executable instructions and/or specializedhardware configured to perform image processing and object recognitionanalyses involved in various embodiments. For example, the image andobject recognition processor 306 may be configured to perform theoperations of processing images received from cameras (e.g., 122, 136)via the CAM 305 to recognize and/or identify other vehicles, andotherwise perform functions of the camera perception layer 204 asdescribed. In some embodiments, the processor 306 may be configured toprocess radar or lidar data and perform functions of the radarperception layer 202 as described.

The system components and resources 316, analog and custom circuitry314, and/or CAM 305 may include circuitry to interface with peripheraldevices, such as cameras 122, 136, radar 132, lidar 138, electronicdisplays, wireless communication devices, external memory chips, etc.The processors 303, 304, 306, 307, 308 may be interconnected to one ormore memory elements 312, system components and resources 316, analogand custom circuitry 314, CAM 305, and RPM processor 317 via aninterconnection/bus module 324, which may include an array ofreconfigurable logic gates and/or implement a bus architecture (e.g.,CoreConnect, AMBA, etc.). Communications may be provided by advancedinterconnects, such as high-performance networks-on chip (NoCs).

The processing device SOC 300 may further include an input/output module(not illustrated) for communicating with resources external to the SOC,such as a clock 318 and a voltage regulator 320. Resources external tothe SOC (e.g., clock 318, voltage regulator 320) may be shared by two ormore of the internal SOC processors/cores (e.g., a DSP 303, a modemprocessor 304, a graphics processor 306, an applications processor 308,etc.).

In some embodiments, the processing device SOC 300 may be included in acontrol unit (e.g., 140) for use in a vehicle (e.g., 100). The controlunit may include communication links for communication with a telephonenetwork (e.g., 180), the Internet, and/or a network server (e.g., 184)as described.

The processing device SOC 300 may also include additional hardwareand/or software components that are suitable for collecting sensor datafrom sensors, including motion sensors (e.g., accelerometers andgyroscopes of an IMU), user interface elements (e.g., input buttons,touch screen display, etc.), microphone arrays, sensors for monitoringphysical conditions (e.g., location, direction, motion, orientation,vibration, pressure, etc.), cameras, compasses, GPS receivers,communications circuitry (e.g., Bluetooth®, WLAN, WiFi, etc.), and otherwell-known components of modern electronic devices.

FIG. 4 illustrates a method 400 of broadcasting an intention messageaccording to various embodiments. With reference to FIGS. 1A-4, themethod 400 may be implemented in a processor (e.g., 164), a processingdevice (e.g., 300), and/or a control unit (e.g., 104) (variouslyreferred to as a “processor”) of a vehicle (e.g., 100). In someembodiments, the method 400 may be performed by one or more layerswithin a vehicle management system stack, such as a vehicle managementstack 200, a vehicle management stack 250, etc. In other embodiments,the method 400 may be performed by a processor independently from, butin conjunction with, a vehicle control system stack, such as a vehiclemanagement stack 200, a vehicle management stack 250, etc. For example,the method 400 may be implemented as a stand-alone software module orwithin dedicated hardware that monitors data and commands from/withinthe vehicle management system stack (e.g., vehicle management stack 200,250, etc.) and is configured to take actions and store data asdescribed.

In block 402, the processor may determine a motion plan for the vehicle.In various embodiments, a motion plan may include a position and anindication of how the position over time to is expected to change. Insome embodiments, a motion plan may include a trajectory. An autonomousor semi-autonomous vehicle may know its current position and thedirection and/or next action the autonomous or semi-autonomous vehiclemay take to arrive at a next position at a next time. For example, theautonomous vehicle may have a trajectory to get to a next point from acurrent point. In various embodiments, a motion plan may include atrajectory and one or more vehicle descriptors associated with thevehicle and/or the vehicle owner and/or operator. For example, vehicledescriptors may include sensor perceptible attributes of the vehicle,such as vehicle color, vehicle license plate number, vehicle size, etc.As a further example, vehicle descriptors may include vehicle physicalcapabilities, such as vehicle type, vehicle turning radius, vehicle topspeed, vehicle maximum acceleration, etc. As a still further example,vehicle descriptors may include vehicle location attributes, such as thevehicle's latitude & longitude, the vehicle's distance from, and/ororientation to, a known landmark in a coordinate plane, etc. In someembodiments, a motion plan may include an indication of an expected nextposition at a certain time. In some embodiments, a motion plan mayinclude a motion vector. In some embodiments, a motion plan may includean indication of the coordinate plane used for determining the indicatedposition. In various embodiments, the motion plan may describe featuresof the vehicle transmitting the motion plan, such as its size,orientation, color, vehicle type, etc. In various embodiments, themotion plan may indicate the speed of the vehicle transmitting themotion plan, orientation of the vehicle transmitting the motion plan,acceleration of the vehicle transmitting the motion plan, or any otherstate information of the vehicle transmitting the motion plan. Invarious embodiments, the motion plan may indicate future actions (orintentions) of the vehicle, such as “turning on left blinker in fiveseconds”, “turning right in two seconds”, “braking in one hundred feet”,or any other type actions or intentions relevant to driving.

In block 404, the processor may generate an intention message based atleast in part on the determined motion plan. In various embodiments, anintention message may include an identifier of the vehicle broadcastingthe intention message, the current position of the vehicle broadcastingthe intention message, the motion plan of the vehicle broadcasting theintention message, and/or other data related to the vehicle broadcastingthe intention message.

In block 406, the processor may broadcast the intention message. Invarious embodiments, intention messages may be broadcast from anautonomous vehicle, such as by C-V2X transmission modes. In variousembodiments, intention messages may be broadcast periodically, such asat a set time interval, in response to a change in intention of abroadcasting vehicle, etc.

FIG. 5 illustrates a method 500 of extracting a motion plan from abroadcast intention message according to various embodiments. Withreference to FIGS. 1A-5, the method 500 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 500 may be performed by oneor more layers within a vehicle management system stack, such as avehicle management stack 200, a vehicle management stack 250, etc. Inother embodiments, the method 500 may be performed by a processorindependently from, but in conjunction with, a vehicle control systemstack, such as a vehicle management stack 200, a vehicle managementstack 250, etc. For example, the method 500 may be implemented as astand-alone software module or within dedicated hardware that monitorsdata and commands from/within the vehicle management system stack (e.g.,vehicle management stack 200, 250, etc.) and is configured to takeactions and store data as described. In various embodiments, theoperations of method 500 may be performed in conjunction with theoperations of method 400 (FIG. 4).

In block 502, the processor may receive an intention message. In variousembodiments, broadcast intention messages may be received by any vehiclewithin transmission range of the vehicles broadcasting such intentionmessages. In various embodiments, the intention message may be receivedvia C-V2X transmissions from a broadcasting vehicle.

In block 504, the processor may parse the intention message to identifythe autonomous (or semi-autonomous) vehicle transmitting the intentionmessage (the “transmitting vehicle”) and a motion plan for thattransmitting vehicle. In various embodiments, a vehicle, such as anautonomous vehicle, a semi-autonomous vehicle, a non-autonomous vehicle,etc., may parse the received intention message to determine thebroadcasting vehicle's identifier, the current position of the intentionmessage, and the motion plan, and/or any other information indicatedwithin the intention message. In various embodiments, the motion planmay include a trajectory and one or more vehicle descriptors associatedwith the vehicle and/or the vehicle owner and/or operator. For example,vehicle descriptors may include sensor perceptible attributes of thevehicle, such as vehicle color, vehicle license plate number, vehiclesize, etc. As a further example, vehicle descriptors may include vehiclephysical capabilities, such as vehicle type, vehicle turning radius,vehicle top speed, vehicle maximum acceleration, etc. As a still furtherexample, vehicle descriptors may include vehicle location attributes,such as the vehicle's latitude & longitude, the vehicle's distance from,and/or orientation to, a known landmark in a coordinate plane, etc.

In block 506, the processor may send the motion plan for thetransmitting vehicle. In various embodiments, the broadcasting vehicle'sidentifier, the current position of the intention message, and themotion plan, and/or any other information indicated within the intentionmessage may be provided to various hardware and software components ofthe receiving vehicle. For example, the broadcasting vehicle'sidentifier, the current position of the intention message, the motionplan, and/or any other information indicated within the intentionmessage may be stored in one or more memory locations on the receivingvehicle, may be sent to one or more layers of a vehicle managementsystem stack, etc.

In block 508, the processor may control the vehicle based at least inpart on the motion plan. For example, using the broadcasting vehicle'sidentifier, the current position of the intention message, the motionplan, and/or any other information indicated within the intentionmessage, the vehicle management system stack may control the operationsof the vehicle. In various embodiments, motion plans may be used by oneor more various layers of the vehicle management system stack to augmentvarious decision making and/or autonomous driving operations. Asexamples, a received motion plan may be used by the vehicle managementsystem stack in: sensor fusion processing; behavior prediction;behavioral planning; motion planning; position localization; and/orsharing the burden of safety operations between autonomous vehicles. Asspecific examples, the motion plan may be used to control the operationsof the vehicle in various embodiment methods described herein (e.g.,method 400 (FIG. 4), method 500 (FIG. 5), method 600 (FIG. 6), method700 (FIG. 7), method 800 (FIG. 8A), method 850 (FIG. 8B), method 900(FIG. 9), and/or method 1000 (FIG. 10)).

FIG. 6 illustrates a method 600 of using a broadcast motion plan insensor perception operations according to various embodiments. Withreference to FIGS. 1A-6, the method 600 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 600 may be performed by oneor more layers within a vehicle management system stack, such as avehicle management stack 200, a vehicle management stack 250, etc. Forexample, some or all of operations of the method 600 may be performed aspart of a perception functions implemented within the camera perceptionlayer 204 and/or radar perception layer 202. In other embodiments, themethod 600 may be performed by a processor independently from, but inconjunction with, the vehicle management system stack, such as thevehicle management stack 200, the vehicle management stack 250, etc. Forexample, the method 600 may be implemented as a stand-alone softwaremodule or within dedicated hardware that monitors data and commandsfrom/within the vehicle management system stack (e.g., vehiclemanagement system stack 200, 250, etc.) and is configured to takeactions and store data as described. In various embodiments, theoperations of method 600 may be performed in conjunction with theoperations of method 400 (FIG. 4) and/or method 500 (FIG. 5). In variousembodiments, the operations of the method 600 may be example operationsto control a vehicle based at least in part on a motion plan.

In block 602, the processor may receive a motion plan. A motion plan maybe received in various manners, such as in a message received fromanother component, retrieving the motion plan from a memory location(e.g., a cache, etc.) used for storing motion plans, in response to arequest for a motion plan, etc. In various embodiments, a motion planmay include a trajectory and one or more vehicle descriptors associatedwith the vehicle and/or the vehicle owner and/or operator. For example,vehicle descriptors may include sensor perceptible attributes of thevehicle, such as vehicle color, vehicle license plate number, vehiclesize, etc. As a further example, vehicle descriptors may include vehiclephysical capabilities, such as vehicle type, vehicle turning radius,vehicle top speed, vehicle maximum acceleration, etc. As a still furtherexample, vehicle descriptors may include vehicle location attributes,such as the vehicle's latitude and longitude, the vehicle's distancefrom, and/or orientation to, a known landmark in a coordinate plane,etc.

In block 604, the processor may determine an expected region of interestfor a vehicle based at least in part on the received motion plan. Beingable to focus on a specific region of interest in the data in the spacearound the vehicle because an object, such as the motion planbroadcasting vehicle, is expected in that region of interest mayincrease the speed of detection of that vehicle when compared withanalyzing the data as a whole. In various embodiments, the motion planmay be used to determine the region of interest to apply to the rawsensor data to perceive a vehicle in that raw sensor data.

In optional block 606, the processor may select a detection algorithmbased at least in part on the received motion plan. In some embodiments,the motion plan may be used to select and/or modify the algorithm usedto perceive vehicles in the raw sensor data. For example, a differentalgorithm may be used when the motion plan broadcasting vehicle isexpected to be head on to the receiving vehicle than when the motionplan broadcasting vehicle is expected to be perpendicular to thereceiving vehicle. As a further example, a different detection thresholdmay be used depending on whether a motion plan indicates thebroadcasting vehicle is expected to be in a given space. As a specificexample, without a motion plan a receiving vehicle may only reportdetections that pass with a 90% confidence, while with a motion plan thereceiving vehicle may report detections that pass a 50% confidence in aregion of interest associated with the motion plan. Block 606 may beoptional, as the detection algorithm may not change in some embodiments.

In block 608, the processor may receive sensor data. The sensor data maybe raw sensor data received from one or more sensors, such as cameras,LIDARs, radars, etc.

In block 610, the processor may apply the detection algorithm to thesensor data at the expected region of interest to detect the vehicle inthe sensor data. Sensor perception may include operations to controlwhere a sensor looks in the region of interest to confirm a detection ofan object, such as another vehicle. A motion plan that indicates vehicledescriptors which sensor perceptible attributes, may enable a vehiclemanagement system to focus one or more sensors on a specific sensorperceptible attribute, such as cueing an image sensor to look for aparticular vehicle color, vehicle size, etc. This may increase the speedof detection by sensors and the vehicle management system of that othervehicle compared to analyzing the entirety of sensor data as a whole. Insome embodiments, the motion plan may be used by the vehicle managementsystem to confirm whether or not a specific vehicle is perceived in theraw sensor data. For example, the detection in raw sensor data of asensor perceptible attribute of a vehicle that was included in a motionplan as a vehicle descriptor (e.g., the vehicle's color, vehicle'slicense plate number, etc.) may confirm that the vehicle transmittingthe motion plan is the vehicle perceived by vehicle sensors in a regionof interest.

In block 612, the processor may send the vehicle detection sensor data.For example, the vehicle detection sensor data may be sent to the sensorfusion and RWM management layer 212.

FIG. 7 illustrates a method 700 of using a broadcast motion plan insensor fusion operations according to various embodiments. Withreference to FIGS. 1A-7, the method 700 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 700 may be performed by oneor more layers within a vehicle management system stack, such as avehicle management system stack 200, a vehicle management system stack250, etc. For example, some or all of operations of the method 700 maybe performed as part of a sensor fusion functions implemented within thesensor fusion and RWM management layer 212. In other embodiments, themethod 700 may be performed by a processor independently from, but inconjunction with, the vehicle management system stack, such as thevehicle management system stack 200, the vehicle management system stack250, etc. For example, the method 700 may be implemented as astand-alone software module or within dedicated hardware that monitorsdata and commands from/within the vehicle management system stack (e.g.,vehicle management system stack 200, 250, etc.) and is configured totake actions and store data as described. In various embodiments, theoperations of method 700 may be performed in conjunction with theoperations of method 400 (FIG. 4), method 500 (FIG. 5), and/or method600 (FIG. 6). In various embodiments, the operations of method 700 maybe example operations to control a vehicle based at least in part on amotion plan.

In block 702, the processor may receive a motion plan. A motion plan maybe received in various manners, such as in a message received fromanother component, retrieving the motion plan from a memory location(e.g., a cache, etc.) used for storing motion plans, in response to arequest for a motion plan, etc. In various embodiments, a motion planmay include a trajectory and one or more vehicle descriptors associatedwith the vehicle and/or the vehicle owner and/or operator. For example,vehicle descriptors may include sensor perceptible attributes of thevehicle, such as vehicle color, vehicle license plate number, vehiclesize, etc. As a further example, vehicle descriptors may include vehiclephysical capabilities, such as vehicle type, vehicle turning radius,vehicle top speed, vehicle maximum acceleration, etc. As a still furtherexample, vehicle descriptors may include vehicle location attributes,such as the vehicle's latitude & longitude, the vehicle's distance fromand/or orientation to a known landmark in a coordinate plane, etc.

In block 704, the processor may receive vehicle detection sensor data.For example, the processor may receive vehicle detection sensor datafrom the camera perception layer 204 and/or radar perception layer 202.

In block 706, the processor may correlate vehicle detection sensor datawith an associated vehicle based at least in part on the received motionplan for that vehicle. In various embodiments, correlating vehicledetection sensor data with an associated vehicle based at least in parton the received motion plan for that vehicle may include operations tocombine and associate sensor data and associate that sensor data with atracked object. A motion plan may improve the performance of dataassociation and tracking operations of a sensor fusion layer of avehicle management system. The vehicle management system may use themotion plan to determine how to fuse all the raw detections of othervehicles in an environment together. For example, based on the motionplan broadcast for a vehicle, the receiving vehicle may be enabled todetermine a radar detection, a LIDAR detection, and a camera detectionare actually all the same vehicle because the detections all correlateto the motion plan for that vehicle rather than considering the threedetections separate vehicles. As a specific example, different sensorsof the receiving vehicle positively identifying one or more sensorperceptible attributes of a vehicle included in a received motion planmay confirm that the sensors are detecting the vehicle that transmittedthe motion plan. Additionally, the motion plan may enable the receivingvehicle to determine that those vehicle detections will evolve together.The ability to compare vehicle detections to motion plans may enableoutlier measurements to be discarded. For example, a tracked vehicle'smotion plan indicating it intends to stay in a current lane may enable adetection not corresponding to that lane to be associated with a newobject rather than the previously tracked vehicle. The presence of amotion plan may reduce uncertainty in the sensor fusion operations. Thenoisy detections from the perception layer may be compared with theunderlying trajectories in a motion plan to give an improved certaintyto sensor fusion operations.

In block 708, the processor may send combined vehicle detection sensordata. For example, the processor may send the combined vehicle detectionsensor data to a behavioral planning and prediction layer 216, vehiclesafety and crash avoidance system 252, and/or motion planning andcontrol layer 214.

FIG. 8A illustrates a method 800 of using a broadcast motion plan inbehavior prediction operations according to various embodiments. Withreference to FIGS. 1A-8A, the method 800 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 800 may be performed by oneor more layers within a vehicle management system stack, such as avehicle management system stack 200, a vehicle management system stack250, etc. For example, some or all of operations of the method 800 maybe performed as part of a behavioral prediction functions implementedwithin the behavioral planning and prediction layer 216. In otherembodiments, the method 800 may be performed by a processorindependently from, but in conjunction with, the vehicle managementsystem stack, such as the vehicle management system stack 200, thevehicle management system stack 250, etc. For example, the method 800may be implemented as a stand-alone software module or within dedicatedhardware that monitors data and commands from/within the vehiclemanagement system stack (e.g., vehicle management system stack 200, 250,etc.) and is configured to take actions and store data as described. Invarious embodiments, the operations of method 800 may be performed inconjunction with the operations of method 400 (FIG. 4), method 500 (FIG.5), method 600 (FIG. 6), and/or method 700 (FIG. 7). In variousembodiments, the operations of method 800 may be example operations tocontrol a vehicle based at least in part on a motion plan.

In block 802, the processor may identify a tracked vehicle. For example,the processor may select a next tracked vehicle in a list of trackedvehicle currently perceived in the environment based on data from thesensor fusion and RWM management layer 212.

In determination block 804, the processor may determine whether themotion plan has been received for the tracked vehicle. For example, theprocessor may compare the tracked vehicle identifier to the identifiersof motion plans stored in a memory to determine whether a motion planhas been received for the tracked vehicle. In some embodiments, when amotion plan is received, the motion plan may include vehicle descriptorsthat reflect vehicle physical capabilities of the vehicle transmittingthe motion plan.

In response to determining that the motion plan has not been received(i.e., determination block 804=“No”), the processor may determine abehavior prediction in block 808. In this manner, even though nointention messages are received, the processor may revert to making thevehicle fully responsible for full inference of the state of the worldby determining a behavior prediction. In a similar manner as describedwith reference to blocks 804 and 808 of the method 800 (FIG. 8A), theother embodiment methods described herein (e.g., method 400 (FIG. 4),method 500 (FIG. 5), method 600 (FIG. 6), method 700 (FIG. 7), method900 (FIG. 9), and/or method 1000 (FIG. 10)), may optionally includeoperations to conditionally take actions (and/or make determinations)based on one or more available motion plans when such broadcast motionplans are available and to default to other actions (and/or make otherdeterminations) when no such broadcast motion plans are received. Invarious embodiments, for example, when no intention messages (or motionplans) are received, the processor may revert to making the vehiclefully responsible for full inference of the state of the world to enabletaking actions (and/or making determinations).

In response to determining that the motion plan has been received (i.e.,determination block 804=“Yes”), the processor may set a behaviorprediction based on the received motion plan in block 806. The presenceof a motion plan broadcast by a vehicle may enable a vehicle receivingthat motion plan to predict the behavior of that vehicle with a knowncertainty. The benefit of a motion plan and a position may be that themotion plan may be treated as the predicted behavior of that vehicle.This certainty in behavior prediction may reduce the dimensionality ofthe POMDP by knowing behavioral/trajectory prediction exactly forsurrounding cars that broadcast their respective motion plans. Sharingthe motion plan may result in behavioral predictions with nouncertainty. Additionally, knowing the intended motion of a vehicle mayeliminate the need to estimate that vehicle's motion, thereby reducingthe computational resources associated with behavior prediction.Additionally, a motion plan including vehicle descriptors that reflectvehicle physical capabilities of the vehicle transmitting the motionplan may reduce the space of possibilities for behavioral planningsearches within a vehicle behavior model by limiting the possiblebehaviors of the transmitting vehicle to those within that vehicle'scapabilities. For example, a type of vehicle indicated in a motion planmay be used to constrain a maximum speed or acceleration of the vehiclebased on the maximum speed or acceleration associated with that vehicletype. As another example, a turning radius of the vehicle indicated in amotion plan may be used to constrain the potential turning paths of thevehicle transmitting the motion plan to within the indicated turningradius.

In optional block 810, the processor may indicate the behaviorprediction as having a high certainty. For example, the certainty may behigh as the broadcasting vehicle affirmatively indicated its intentionin its motion plan and the behavior prediction was set to that indicatedintention.

In various embodiments, behavior predictions based on the motion plan ofa broadcasting vehicle may be used for behavioral planning and/or motionplanning with a high degree of certainty. In various embodiments, abehavioral planning layer may provide a high-level driving goal to amotion planning layer. The motion planning layer may be responsible foractually planning the trajectory to execute that high-level maneuver andthe motion planning layer may be responsible for ensuring the safety ofexecuting that trajectory. The presence of a motion plan broadcast byanother vehicle in the environment may enable the motion planning layerto perform fewer collision checks as a result of prediction of themotion of that broadcasting vehicle being known with certainty.Collision checking is often a bottleneck of fast motion planning, sofewer checks may greatly speed up the overall autonomous drivingalgorithm.

FIG. 8B illustrates a method 850 of using a broadcast motion plan inbehavior prediction operations according to various embodiments. Withreference to FIGS. 1A-8B, the method 850 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 850 may be performed by oneor more layers within a vehicle management system stack, such as avehicle management system stack 200, a vehicle management system stack250, etc. For example, some or all of operations of the method 850 maybe performed as part of behavioral prediction functions implementedwithin the behavioral planning and prediction layer 216. In otherembodiments, the method 850 may be performed by a processorindependently from, but in conjunction with, the vehicle managementsystem stack, such as the vehicle management system stack 200, thevehicle management system stack 250, etc. For example, the method 850may be implemented as a stand-alone software module or within dedicatedhardware that monitors data and commands from/within the vehiclemanagement system stack (e.g., vehicle management system stack 200, 250,etc.) and is configured to take actions and store data as described. Invarious embodiments, the operations of method 850 may be performed inconjunction with the operations of method 400 (FIG. 4), method 500 (FIG.5), method 600 (FIG. 6), and/or method 700 (FIG. 7). In variousembodiments, the operations of the method 850 may be example operationsto control a vehicle based at least in part on a received motion plan.

In blocks 802, 804, 806, 808, and 810, the processor may perform likeoperations of like numbered blocks of method 800 (FIG. 8A) describedabove.

In determination block 852, the processor may determine whether thebehavior of the tracked other vehicle conformed with the behaviorprediction by the behavioral planning and prediction layer 216. Forexample, the processor may compare sensor data associated with thetracked vehicle to the behavior prediction by the behavioral planningand prediction layer to determine whether the tracked vehicle is at anexpected location per the behavior prediction. The sensor dataindicating the tracked vehicle is located where expected may indicatethat the behavior of the tracked vehicle conformed to the behaviorprediction. The sensor data indicating the tracked vehicle is notlocated where expected may indicate that the behavior of the trackedvehicle did not conform to the behavior prediction.

In response to determining that the behavior conformed to the behaviorprediction (i.e., determination block 852=“Yes”), the processor maycontinue to monitor the behavior of the tracked vehicle and continuouslyor periodically determine whether the behavior conformed to the behaviorprediction in determination block 852. In this manner, processor mayrepeatedly check for non-conforming behavior of the tracked vehicle.

In response to determining that the behavior did not conform to thebehavior prediction (i.e., determination block 852=“No”), the processormay update the behavior prediction or the vehicle behavior model basedon information in the received motion plan in block 854. In variousembodiments, receiving a motion plan broadcast by a vehicle includingvehicle descriptors that reflect vehicle physical capabilities of thevehicle transmitting the motion plan may reduce the space ofpossibilities for behavioral planning searches by or within a vehiclebehavior model after a vehicle deviates from a behavior prediction. Inresponse to determining that a vehicle's behavior does not conform to abehavior prediction, the behavior prediction or the vehicle behaviormodel for that vehicle may be updated based at least in part oninformation in the received motion plan. For example, vehicle physicalcapabilities identified in the received motion plan may be used by orwithin a vehicle behavior model to collapse the possible futurebehaviors of the vehicle from a POMDP to an MDP analysis in which thereare a host of solutions suitable for online and real-time operations.For example, a type of vehicle indicated in a motion plan may be used toconstrain a maximum speed or acceleration of the vehicle used in avehicle behavior model that is then used to update the behaviorprediction based on the maximum speed or acceleration associated withthat vehicle type. As another example, a turning radius of the vehicleindicated in a motion plan may be used in a vehicle behavior model toconstrain the potential turning paths of the vehicle transmitting themotion plan to within the indicated turning radius used in updating thebehavior prediction.

FIG. 9 illustrates a method 900 of using a broadcast motion plan inposition localization operations according to various embodiments. Withreference to FIGS. 1A-9, the method 900 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 900 may be performed by oneor more layers within a vehicle management system stack, such as avehicle management system stack 200, a vehicle management system stack250, etc. For example, some or all of operations of the method 900 maybe performed as part of a localization functions implemented within themap fusion and arbitration layer 208. In other embodiments, the method900 may be performed by a processor independently from, but inconjunction with, the vehicle management system stack, such as thevehicle management system stack 200, the vehicle management system stack250, etc. For example, the method 900 may be implemented as astand-alone software module or within dedicated hardware that monitorsdata and commands from/within the vehicle management system stack (e.g.,vehicle management system stack 200, 250, etc.) and is configured totake actions and store data as described. In various embodiments, theoperations of method 900 may be performed in conjunction with theoperations of method 400 (FIG. 4), method 500 (FIG. 5), method 600 (FIG.6), method 700 (FIG. 7), method 800 (FIG. 8A), and/or method 850 (FIG.8B). In various embodiments, the operations of method 900 may be exampleoperations to control a vehicle based at least in part on a motion plan.

In block 902, the processor may receive one or more motion plans. Motionplans may be received in various manners, such as in messages receivedfrom another component, retrieving the motion plans from a memorylocation (e.g., a cache, etc.) used for storing motion plans, inresponse to a request for motion plans, etc. In various embodiments, amotion plan may include a trajectory and one or more vehicle descriptorsassociated with the vehicle and/or the vehicle owner and/or operator asdescribed.

In block 904, the processor may determine positions of other vehiclesbased on their respective motion plans. In various embodiments,intention messages and/or motion plans broadcast by vehicles mayindicate those vehicles distance from, and/or orientation to, a knownlandmark in a coordinate plan, such as a local and/or global coordinateplane. For example, a motion plan may indicate the transmittingvehicle's distance from and/or orientation to a known landmark in acoordinate plane, such as a local and/or global coordinate plane, in avehicle location attribute type vehicle descriptor. The processor maydetermine the positions and/or relative distances from a known landmarkfor the other vehicles broadcasting motion plans to determine positionsof other vehicles based on their respective motion plans.

In determination block 906, the processor may determine whether acomparison between the vehicle's own position and other vehiclepositions indicates an error. A vehicle, such as an autonomous vehicle,a semi-autonomous vehicle, a non-autonomous vehicle, etc., receiving theindications from the broadcasting vehicles may compare its observationsof those broadcasting vehicles and their position relative to the knownlandmark to the distance from the known landmark in the intentionmessage and/or motion plan to assist in localization of the receivingvehicle. For example, the receiving vehicle may compare its ownobservations to those in the motion plans to determine whether there isan error or offset between its observations and those in the motionplans. An error or offset between its observations and those in themotion plans may indicate to the receiving vehicle that it has localizedits respective current position incorrectly. As a specific example, twodifferent autonomous vehicles may both broadcast their respective motionplans and those motion plans may be received by the receiving vehicle.Observations of a landmark in those two motion plans may match, but maybe different from the observation of the receiving vehicle of that samelandmark. The agreement of two observations in the different motionplans and their difference from the receiving vehicle's observations mayindicate the receiving vehicle localized its position incorrectly.

In response to no error being indicated (i.e., determination block906=“No”), the processor may continue to receive motion plans in block902.

In response to an error being indicated by the comparison between thevehicle's own position and other vehicle positions (i.e., determinationblock 906=“Yes”), the processor may trigger a recalculation of thevehicle's own position in block 908.

FIG. 10 illustrates a method 1000 of using a broadcast motion plan toshare the burden of safety operations according to various embodiments.With reference to FIGS. 1A-10, the method 1000 may be implemented in aprocessor (e.g., 164), a processing device (e.g., 300), and/or a controlunit (e.g., 104) (variously referred to as a “processor”) of a vehicle(e.g., 100). In some embodiments, the method 1000 may be performed byone or more layers within a vehicle management system stack, such as avehicle management system stack 200, a vehicle management system stack250, etc. For example, some or all of operations of the method 1000 maybe performed as part of a safety functions implemented within the motionplanning and control layer 214. In other embodiments, the method 1000may be performed by a processor independently from, but in conjunctionwith, the vehicle management system stack, such as the vehiclemanagement system stack 200, the vehicle management system stack 250,etc. For example, the method 1000 may be implemented as a stand-alonesoftware module or within dedicated hardware that monitors data andcommands from/within the vehicle management system stack (e.g., vehiclemanagement system stack 200, 250, etc.) and is configured to takeactions and store data as described. For example, some or all ofoperations of the method 1000 may be performed as part of a safetyfunctions implemented within the vehicle safety and crash avoidancesystem 252. In various embodiments, the operations of method 1000 may beperformed in conjunction with the operations of method 400 (FIG. 4),method 500 (FIG. 5), method 600 (FIG. 6), method 700 (FIG. 7), method800 (FIG. 8), and/or method 900 (FIG. 9). In various embodiments, theoperations of method 1000 may be example operations to control a vehiclebased at least in part on a received motion plan.

In block 1002, the processor may receive a motion plan. A motion planmay be received in various manners, such as in a message received fromanother component, retrieving the motion plan from a memory location(e.g., a cache, etc.) used for storing motion plans, in response to arequest for a motion plan, etc. In various embodiments, a motion planmay include a trajectory and one or more vehicle descriptors associatedwith the vehicle and/or the vehicle owner and/or operator as described.

In determination block 1006, the processor may determine whether themotion plan is unsafe. For example, the receiving vehicle may detect anobject that will cause a motion plan of a broadcasting vehicle to beunsafe even though the broadcasting vehicle did not yet sense orotherwise observe that object. In response, the receiving vehicle maythat determine the motion plan is unsafe.

In response to determining that the motion plan is safe (i.e.,determination block 1006=“No”), the processor may take no action inblock 1010.

In response to determining that the motion plan is unsafe (i.e.,determination block 1006=“Yes”), the processor may send a safety warningto the other vehicle in block 1008. The warning may include theobservation of the object causing the motion plan to be unsafe.

Various embodiments illustrated and described are provided merely asexamples to illustrate various features of the claims. However, featuresshown and described with respect to any given embodiment are notnecessarily limited to the associated embodiment and may be used orcombined with other embodiments that are shown and described. Further,the claims are not intended to be limited by any one example embodiment.

The foregoing method descriptions and the process flow diagrams areprovided merely as illustrative examples and are not intended to requireor imply that the blocks of various embodiments must be performed in theorder presented. As will be appreciated by one of skill in the art theorder of blocks in the foregoing embodiments may be performed in anyorder. Words such as “thereafter,” “then,” “next,” etc. are not intendedto limit the order of the blocks; these words are simply used to guidethe reader through the description of the methods. Further, anyreference to claim elements in the singular, for example, using thearticles “a,” “an” or “the” is not to be construed as limiting theelement to the singular.

The various illustrative logical blocks, modules, circuits, andalgorithm blocks described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and blocks have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such embodiment decisions should not beinterpreted as causing a departure from the scope of variousembodiments.

The hardware used to implement the various illustrative logics, logicalblocks, modules, and circuits described in connection with theembodiments disclosed herein may be implemented or performed with ageneral-purpose processor, a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA) or other programmable logic device, discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. Ageneral-purpose processor may be a microprocessor, but, in thealternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of communication devices, e.g., acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Alternatively, some blocks ormethods may be performed by circuitry that is specific to a givenfunction.

In various embodiments, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored as one or more instructions orcode on a non-transitory computer-readable medium or non-transitoryprocessor-readable medium. The operations of a method or algorithmdisclosed herein may be embodied in a processor-executable softwaremodule, which may reside on a non-transitory computer-readable orprocessor-readable storage medium. Non-transitory computer-readable orprocessor-readable storage media may be any storage media that may beaccessed by a computer or a processor. By way of example but notlimitation, such non-transitory computer-readable or processor-readablemedia may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that may be used to store desired programcode in the form of instructions or data structures and that may beaccessed by a computer. Disk and disc, as used herein, includes compactdisc (CD), laser disc, optical disc, digital versatile disc (DVD),floppy disk, and Blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above are also included within the scope ofnon-transitory computer-readable and processor-readable media.Additionally, the operations of a method or algorithm may reside as oneor any combination or set of codes and/or instructions on anon-transitory processor-readable medium and/or computer-readablemedium, which may be incorporated into a computer program product.

The preceding description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentembodiments. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thescope of the embodiments. Thus, various embodiments are not intended tobe limited to the embodiments shown herein but are to be accorded thewidest scope consistent with the following claims and the principles andnovel features disclosed herein.

What is claimed is:
 1. A method of controlling a vehicle, comprising:receiving an intention message including a motion plan for a vehicletransmitting the motion plan (the “transmitting vehicle”), wherein themotion plan comprises a trajectory of the transmitting vehicle and oneor more vehicle descriptors associated with the transmitting vehicle;parsing the intention message to identify the motion plan for thetransmitting vehicle; and controlling the vehicle based at least in parton the motion plan.
 2. The method of claim 1, wherein the one or morevehicle descriptors comprise a sensor perceptible attribute.
 3. Themethod of claim 2, wherein controlling the vehicle based at least inpart on the motion plan comprises: determining an expected region ofinterest for the vehicle based at least in part on the motion plan; andapplying a detection algorithm to received sensor data at the expectedregion of interest to detect the transmitting vehicle in the receivedsensor data based at least in part on the sensor perceptible attribute.4. The method of claim 3, further comprising selecting the detectionalgorithm based at least in part on the received motion plan.
 5. Themethod of claim 2, wherein controlling the vehicle based at least inpart on the motion plan comprises: correlating vehicle detection sensordata with the transmitting vehicle based at least in part on the sensorperceptible attribute.
 6. The method of claim 1, wherein the one or morevehicle descriptors comprise a vehicle physical capability.
 7. Themethod of claim 6, wherein controlling the vehicle based at least inpart on the motion plan comprises: setting a behavior prediction for thetransmitting vehicle based at least in part on the motion plan.
 8. Themethod of claim 7, further comprising: determining whether a behavior ofthe transmitting vehicle conforms to the behavior prediction; andupdating the behavior prediction based at least in part on the vehiclephysical capability in response to determining that the behavior of thetransmitting vehicle does not conform to the behavior prediction.
 9. Themethod of claim 1, wherein the one or more vehicle descriptors comprisea vehicle location attribute.
 10. The method of claim 9, whereincontrolling the vehicle based at least in part on the motion plancomprises: determining a position of the transmitting vehicle based atleast in part on the vehicle location attribute; determining whether acomparison between a position of the vehicle and the position of thetransmitting vehicle indicate an error; and triggering a recalculationof the position of the vehicle in response to determining the comparisonbetween the position of the vehicle and the position of the transmittingvehicle indicate an error.
 11. The method of claim 1, whereincontrolling the vehicle based at least in part on the motion plancomprises: determining whether the motion plan is unsafe; and sending asafety warning to the transmitting vehicle in response to determiningthe motion plan is unsafe.
 12. A method for broadcasting a message froma vehicle, comprising: determining a motion plan for the vehicle,wherein the motion plan comprises a trajectory of the vehicle and one ormore vehicle descriptors of the vehicle; generating an intention messagebased at least in part on the determined motion plan; and broadcastingthe intention message from the vehicle.
 13. The method of claim 12,wherein the one or more vehicle descriptors comprise a sensorperceptible attribute, a vehicle physical capability, or a vehiclelocation attribute.
 14. A processing device for use in a vehicle, theprocessing device configured to: receive an intention message includinga motion plan for a vehicle transmitting the motion plan (the“transmitting vehicle”), wherein the motion plan comprises a trajectoryof the transmitting vehicle and one or more vehicle descriptorsassociated with the transmitting vehicle; parse the intention message toidentify the motion plan for the transmitting vehicle; and control thevehicle based at least in part on the motion plan.
 15. The processingdevice of claim 14, wherein the one or more vehicle descriptors comprisea sensor perceptible attribute.
 16. The processing device of claim 15,wherein the processing device is configured to control the vehicle basedat least in part on the motion plan by: determining an expected regionof interest for the vehicle based at least in part on the motion plan;and applying a detection algorithm to received sensor data at theexpected region of interest to detect the transmitting vehicle in thereceived sensor data based at least in part on the sensor perceptibleattribute.
 17. The processing device of claim 16, wherein the processingdevice is further configured to select the detection algorithm based atleast in part on the received motion plan.
 18. The processing device ofclaim 15, wherein the processing device is configured to control thevehicle based at least in part on the motion plan by: correlatingvehicle detection sensor data with the transmitting vehicle based atleast in part on the sensor perceptible attribute.
 19. The processingdevice of claim 14, wherein the one or more vehicle descriptors comprisea vehicle physical capability.
 20. The processing device of claim 19,wherein the processing device is configured to control the vehicle basedat least in part on the motion plan by: setting a behavior predictionfor the transmitting vehicle based at least in part on the motion plan.21. The processing device of claim 20, wherein the processing device isfurther configured to: determine whether a behavior of the transmittingvehicle conforms to the behavior prediction; and update the behaviorprediction based at least in part on the vehicle physical capability inresponse to determining that the behavior of the transmitting vehicledoes not conform to the behavior prediction.
 22. The processing deviceof claim 14, wherein the one or more vehicle descriptors comprise avehicle location attribute.
 23. The processing device of claim 22,wherein the processing device is configured to control the vehicle basedat least in part on the motion plan by: determining a position of thetransmitting vehicle based at least in part on the vehicle locationattribute; determining whether a comparison between a position of thevehicle and the position of the transmitting vehicle indicate an error;and triggering a recalculation of the position of the vehicle inresponse to determining the comparison between the position of thevehicle and the position of the transmitting vehicle indicate an error.24. The processing device of claim 14, wherein the processing device isconfigured to control the vehicle based at least in part on the motionplan by: determining whether the motion plan is unsafe; and sending asafety warning to the transmitting vehicle in response to determiningthe motion plan is unsafe.
 25. A processing device for use in a vehicle,the processing device configured to: determine a motion plan for thevehicle, wherein the motion plan comprises a trajectory of the vehicleand one or more vehicle descriptors of the vehicle; generate anintention message based at least in part on the determined motion plan;and broadcast the intention message from the vehicle.
 26. The processingdevice of claim 25, wherein the one or more vehicle descriptors comprisea sensor perceptible attribute, a vehicle physical capability, or avehicle location attribute.